RF04: CONTRASTING COMPARATIVE RESEARCH ON 
TEACHING AND LEARNING IN MATHEMATICS 



Coordinators: Jonas Emanuelsson, Goteborg University and 
David Clarke, University of Melbourne 

International comparative research in mathematics education is a growing field. 
Experiences firom recent and ongoing studies seem to have huge impact on both the 
field ofi research and the fiield ofi practise. The very idea ofi both grasping and making 
use ofi diversity lies in the heart ofi all comparative approaches. However there is an 
ongoing need fior enlightened discussion on how the character ofi these results relate 
to the research methods and techniques used and the theoretical and analytical 
perspectives enacted in the research. The main fiocus ofi the fiorum is how these 
dififierent comparative approaches, and the consequent and profiound dififierences in 
project outcomes, can infiorm our individual and collective ways ofi understanding 
learning and teaching in mathematics. 

GENERAL FRAMEWORK 

The idea is to contrast and discuss different approaches, and to discuss both 
differences and similarities, especially in the character of what we can learn about the 
learning and teaching of mathematics in classrooms from these studies. What are the 
possibilities and limitations associated with different approaches? The different types 
of comparative research that are represented in this forum are: 

OECD-PISA, Organisations doing large scale studies with questionnaires and tests 
lEA-TIMSS, Organisations doing large scale video studies 
EPS, Researchers doing large studies on their own initiatives 
Small scale comparative studies 

SIMILARITY AND DIFFERENCE IN INTERNATIONAL COMPARATIVE 
RESEARCH 

Schmidt, McKnight, Valverde, Houang and Wiley (1997) investigated the 
mathematics curricula of the “almost 50” countries participating in the Third 
International Mathematics and Science Study (TIMSS). The documented differences 
in curricular organisation were extensive. Even within a single country differentiated 
curricular catered to communities perceived as having different needs. Countries 
differed in the extent of such differentiation, in the complexity or uniformity of their 
school systems, and in the distribution of educational decision-making responsibility 
within those school systems. Given such diversity, the identification of any curricular 
similarity with regard to mathematics should be seen as significant. And there were 
significant similarities. There were similarities of topic, if not of curricular location; 
broad correspondences of grade level and content that became differences if you 
looked more closely; differences in the range of content addressed at a particular 
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grade level, but which repeated particular developmental sequences where common 
content was addressed over several grade levels. In another international study of 
mathematics curricula, the OECD study of thirteen countries’ innovative programs in 
mathematics, science and technology found that, “Virtually everywhere, the 
curriculum is becoming more practical” (Atkin & Black, 1997, p. 24). Yet, despite 
this common trend, the same study found significant differences in the reasons that 
prompted the new curricula (Atkin & Black, 1996). These interwoven similarities and 
differences are the signature of international comparative research in mathematics 
education (Clarke, 2003). 

Schmidt, McKnight, Valverde, Houang, and Wiley (1997) reported that differences in 
the characterization of mathematical activity were extreme at the Middle School 
level; from ‘representing’ situations mathematically, ‘generalizing’ and ‘justifying’ to 
‘recalling mathematical objects and properties’ and ‘performing routine procedures.’ 
Despite the apparent diversity, it was the latter two expectations that were 
emphasised in the curricula studied. Given the documented diversity, it is the 
occurrence of similarity that requires explanation. Some curricular similarities may 
be the heritage of a colonial past. Others may be the result of more recent cultural 
imperialism or simply good international marketing. 

In attempting to tease out the patterns of institutional structure and policy evident in 
international comparative research (particularly in the work of LeTendre, Baker, 
Akiba, Goesling, and Wiseman, 2001), Anderson-Levitt (2002) noted the “significant 
national differences in teacher gender, degree of specialization in math, amount of 
planning time, and duties outside class” (p. 19). But these differences co-exist with 
similarities in school organization, classroom organization, and curriculum content. 
Anderson-Levitt (2002, p. 20) juxtaposed the statement by LeTendre et al. that 
“Japanese, German and U.S. teachers all appear to be working from a very similar 
‘cultural script’” (2001, p. 9) with the conclusions of Stigler and Hiebert (1999) that 
U.S. and Japanese teachers use different cultural scripts for running lessons. The 
apparent conflict is usefully (if partially) resolved by noting with Anderson, Ryan 
and Shapiro (1989) that both U.S. and Japanese teachers draw on the same small 
repertoire of “whole-class, lecture-recitation and seatwork lessons conducted by one 
teacher with a group of children isolated in a classroom” (Anderson-Levitt, 2002, 
p.21), but they utilise their options within this repertoire differently. 

LeTendre, Baker, Akiba, Goesling and Wiseman (2001) claim that “Policy debates in 
the U.S. are increasingly informed by use of internationally generated, comparative 
data” (p.3). LeTendre and his colleagues go on to argue that criticisms of 
international comparative research on the basis of “culture clash” ignore international 
isomorphisms at the level of institutions (particularly schools). LeTendre et al. report 
yet another interweaving of similarity and difference. 

We find some differences in how teachers’ work is organised, but similarities in teachers’ 

belief patterns. We find that core teaching practices and teacher beliefs show little 
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national variation, but that other aspects of teachers’ work (e.g., non-instructional duties) 
do show variation (LeTendre, Baker, Akiba, Goesling & Wiseman, 2001, p. 3) 

These differences and the similarities are interconnected and interdependent and it is 
likely that policy and practice are best informed by research that examines the nature 
of the interconnection of specific similarities and differences, rather than simply the 
frequency of their occurrence. This Forum uses brief presentations relating to five 
different research projects, each representing a very different approach to 
international comparative research in mathematics education, as a catalyst for 
discussion of how such research might best inform theory and practice in 
mathematics teaching and learning. 

KEY QUESTIONS 

What can be said about the teaching and learning of mathematics in our own 
countries and how can results be used to reach better performance within our own 
educational systems? 

We have invited researchers that are responsible for very different studies that draw 
on different paradigms and use different methodological approaches. Furthermore, in 
order to give a background to the overarching question above, each contribution will 
address the following questions in relation to their respective study. 

What are the goals of the various international comparative studies? 

By studying reports and other documents from the studies above we see different 
aims in comparing countries. Why do we do it? Is it an effort in trying to find good 
examples of teaching or organisational aspects such as “Lesson study” and 
implement them in our own country? Are other countries’ practises used as mirrors in 
the quest of trying to understand the practise of our own country? These two 
approaches can be related to different ways of interpreting your data. Flence 
producing results of different character. 

What is being studied and how does this relate to teaching and learning? 

The object of research varies between studies. The “what” we are trying to 
understand can be exemplified with: Lesson structure, teacher scripts, , negotiation of 
meaning, object of learning, patterns of interaction and learner practises. 

What are the methods of data collection and analysis employed and with what 
adequacy do they document teaching and learning and their interrelationship? 

The perspectives we adopt in our interpretations of these objects also varies. E.g. 
some studies take their point of departure in the students’ perspective others in the 
teachers’. Furthermore, the theoretical positions are different. They vary both in type 
(pragmatic, socio-cultural, constructivist, phenomenographical) and in explicitness. 
The methods and techniques used in producing data vary considerably. Among the 
group doing classroom research we find examples of studies using audiotape only 
and some use video recording. Among those using video recording, the number of 
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cameras used varies between one and up to three. Other studies use interviews both 
as a principal source of information and as a complement to video recordings. This is 
also true with regard to the use of test and questionnaires as well. There are studies 
where test and/or questionnaires are the only way of collecting data, in others they 
are used to collect supplementary information. 

GOALS 

The forum is intended to deepen the discussion on international comparative studies 
in mathematics education and their potential contribution to theorising mathematics 
teaching and learning. This Forum aims to problematise some of the more superficial 
readings of international comparative research in mathematics education (e.g. league 
tables of national performance) and move discussion within the community towards a 
collective and qualitatively more sophisticated reading and utilisation of the results of 
current and recent comparative studies. Those of us concerned with advancing theory 
in regard to mathematics teaching and learning must develop strategies to realise the 
potential of international comparative research in mathematics education to enhance 
both theory and practice, both in research and in our educational systems. 
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WHAT IS COMPARED IN COMPARATIVE STUDIES OF 
MATHEMATICS EDUCATION? 

Sverker Lindblad and Ference Maiton 



Gothenburg University 



INTRODUCTION 

Our aim is to discuss what is compared in international comparisons in Mathematics 
Education. It goes without saying that what is compared constrains what conclusions 
that are possible to draw from these comparisons. More precisely, presumptions 
about the phenomenon in focus govern our theoretical understanding as well as the 
qualities of facts that are collected. That is trivial from a scientific point of view, but 
not trivial when dealing with comparative studies in Maths Education. In order to 
penetrate Maths Education comparisons we need to describe what is compared in 
well known and significant comparative studies in mathematics education. We have 
chosen the PISA studies, the TIMSS studies and the TIMSS-R studies. 

WHAT IS COMPARED? 

In most international comparisons of Mathematics Education (ME) it is achievement 
in terms of test results that is compared. From such outcome comparisons we can 
conclude that students in some countries are doing better than students in other 
countries. Why this is the case is impossible to tell without further information. But 
we might also collect data about the prerequisites for learning mathematics, such as 
the size of per student investments in education in different countries, class size, 
number of hours in mathematics teaching etc. If the correlation between achievement 
and prerequisites variables, like those above, were high, we could possibly come up 
with conjecture, such as one country could boost achievement in mathematics by 
increasing its investments in education, reducing class size, increasing the number of 
class hours etc. But such correlation evidence is extremely scarce. If outcome 
comparisons have such limitations the next move is in a way self-evident, since we 
need to know what is happening in the teaching process in order to understand the 
outcomes of this process. And this was exactly what the TIMSS-99 did in the most 
advanced attempt to produce plausible explanations of differences in Maths 
achievement between different countries. One hundred year 8 classes were selected 
by random sampling in seven countries. In each class one lesson was video-recorded. 
When all the data were collected and analysed the results were published on the 
internet. We could compare the different countries with regard to, for instance: 
Length of lesson. Time devoted to mathematical work. Time devoted to problem 
segments. Percentage of time devoted to independent problems. Time per 
independent problem. Time devoted to practising new content. Time devoted to 
public interaction. Number of problems assigned as homework. Number of outside 
interruption. Number of problems of moderate complexity. Number of problems that 
included proofs. Number of problems using real life connections. Number of 
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problems requiring the students to make connections, Time devoted to repeating 
procedures, Number of words said by teacher. Number of words said by students. 
Number of lessons during which chalkboard was used. Number of lessons during 
which computational calculators was used. Now, it would not be unreasonable to 
expect several of such factors be correlated with differences in achievement between 
countries, given that more or less the same Mathematical content has been covered in 
different countries. But as should be obvious from table 1 below, this was not the 
case. This means that not only factors like those presented above, referring to how 
Mathematics is taught varied between the countries but also that the content covered, 
i.e. what was taught in Mathematics varied between the countries as well. This in turn 
means that the characteristics of ME referred to different things in different ways. 
Small wonder that basically no correlations with achievement were found! 



TABLE 1 (4.1. in original) Average percentage of problems per eighth-grade 
mathematics lesson within each major category and sub-category topic area, by 
country: TIMSS-1999 (http://nces.ed.gOv/pubs2003/2003013.pdf) p 69. 



Topic area 


AU 


CZ 


HK 


JP2 


NL 


SW 


US 


Number 


36 


27 


18 




16 


42 


30 


Whole numbers, fractions, decimals 


15 


13 


5 




6 


20 


17 


Ratio, proportion, percent 


19 


4 


10 


+ 


6 


19 


6 


Integers 


2 


9 


3 


+ 


4 


3 


8 


Geometry 


29 


26 


24 


84 


32 


33 


22 


Measurement (perimeter and area) 


10 


6 


3 


11 


9 


12 


13 


Two-dimensional geometry 
(polygons, angles, lines) 


14 


15 


17 


73 


15 


17 


4 


Three-dimensional Geometry 


5 


6 


5 


+ 

+ 


9 


4 


5 


Statistics 


9 


3 


2 


+ 


10 


2 


6 


Algebra 


22 


43 


40 


12 


41 


22 


41 


Linear expressions 


7 


16 


11 


J 


6 


5 


6 


Solutions and graphs of linear 
equations and inequalities 


15 


21 


23 


12 


33 


14 


27 


Higher-order functions 


6 


6 


t 


3 


3 


8 


J 


Trigonometry 


+ 


+ 

+ 


14 


+ 


+ 

+ 


+ 

+ 


+ 

+ 


Other 




1 


+ 

+ 




+ 


1 


1 



^Reporting standards not met. Too few cases to be reported. AU=Australia; 
CZ=Czech Republic; HK=Hong Kong SAR; JP=Japan; NL=Netherlands; 
SW=Switzerland; and US=United States. 
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COMPARING DIFFERENT WAYS OF DEALING WITH THE SAME THING 



We believe that these results are extremely important because they show what is not 
critical. So what is critical? 

In order to find pedagogically interesting correlations between what the students 
learn (outcomes) and what happens in the classroom (teaching process), we would 
need to keep one of the two aspects of teaching invariant and making statements 
about e.g. how the same thing was taught (instead of making statements how the 
different things were taught. As a matter of fact Karen Giwen - using data from 
TIMSS-99 - gives, in this research forum, an excellent example that while 
differences between countries in the frequency of a certain category (problems 
requiring students to make connections) was not correlated with differences in 
achievement (for instance, while the best performing country, Hong Kong, had the 
lowest frequency of such problems; the next best performing country, Japan had the 
highest frequency) , differences in the way the same kind of problem were dealt with 
was correlated with differences between countries in achievement. 

Another example of keeping the what-aspect, or the object of learning invariant and 
looking at differences in how the same object of learning is dealt with and thus 
producing pedagogically interesting characterizations of differences between 
classrooms, we see in Ulla Runesson’s and Ida Mok’s contribution to this 
symposium. They found that in the Hong Kong class in their study more different 
aspects of fractions were taught at the same time compared to Swedish classrooms, 
where fewer aspects were brought out in a more sequential (than simultaneous) 
manner. This means that there were different things possible to learn in the Hong 
Kong classroom as compared to the Swedish classrooms. 

We take a third example from our Learner’s Perspective Study. The Swedish team 
was interested in making comparisons of ME in different countries when the same 
topic was taught. Emanuelsson & Sahlstrom (2004) compared, for instance, how the 
geometrical representation of functions was introduced in a Swedish and in an 
American class. 

In the Swedish class the teacher presents three equations: 

K=15x;K=10x, K = 2x 

and the corresponding graphs. The three lines are drawn in three different colours and 
the three functions are written in colours corresponding to the graphs. 

Here, the contrast is between the lines (notably their steepness) and the corresponding 
differences between the written expressions (notably the coefficient of inclination). It 
is basically this correspondence that is possible for the students to learn (and even 
this was made more difficult by using matching colours of equations and lines). 
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Transcript 1. Student initiatives in relation to a teacher question. 

1 . Teacher: WHAT CAN YOU in other words find out from that term or that- that 

number that stands in relation to X in a fonnula like this (3.5) 

2. Student: what 

3. Student: what 

4. Student: what are you on about 

5. Student: what 

6. Students: ((laughs)) 

7. Teacher: we have (.) three formulas 

8. Martina: yes 

9. Teacher: what can one then say on fifteen (.) ten (.) two. (.) directly when you 

get a such formula then you can say something about these here (.) 
anyway on their mutual relation ((points to graph)) (1.0) 

10. Student: eh 

1 1 . Martina: in what order they are or 

12. Teacher: yes- no not what order but how they slope 

13. Johan: high low or eh ... in between 

14. Teacher: sure (.) 



In this case we are seeing the mathematical content as something that is negotiated in 
classroom interaction. Thus, in order to understand the meaning of the content we 
need to understand the meaning(s) of classroom interaction. In the US class the 
students are expected to “compare and contrast”, i.e. to describe similarities and 
differences within five pairs of equations: y=3x-i-2 and y=-3x-2; 0x-i-3y=6 and 
2x-i-0y=6; y=x^ and y=l/x; y=l-2x and y=l-x^ ; 2y=x and y=2x. 

Also in this case the students have the opportunities of learning about the 
correspondence between slope and coefficient of inclination but this relationship is 
extended to the case of zero slope (horizontal lines) and no slope (vertical lines). 
Furthermore the students have the opportunity of forming the concept of linear 
functions through the contrast with non-linear functions. So to the extent that the 
students learn different things in the Swedish and in the American class, the simplest 
explanation would be that their opportunities to learn these things have differed. 



CONCLUDING COMMENTS 

The close relationship between what the students have the opportunity to learn and 
what they actually learn is logically necessary and has also been empirically 
demonstrated (Marton & Morris, 2002; Marton, Tsui et al, 2004) and this relationship 
has been taken as a point of departure for improving learning in ME. By finding out 
the necessary conditions for a certain group of students to appropriate a certain object 
of learning and by bringing those necessary conditions about the likelihood of 
learning is most considerably enhanced (Lo, Marton, Pang & Pong, 2004). So, if we 
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want to understand differences in achievement in ME between students in different 
countries we must explore to what extent the objects of learning reflected in the 
achievement test have been possible at all to appropriate. And in order to do so we 
have to look at how the same objects of learning have been handled in classes in 
different countries. Now, to us it seems that the international comparative studies 
such as TIMSS-R och TIMSS are not designed to be comparative in essence, since 
they show little interest in e.g. keeping the content invariant. So what are they then? 
We put forwards two kinds of understanding. First, an important side-effect is 
making of what is important in Maths Education by means of the items that are used 
in order to measure knowledge in Mathematics. International comparisons such as the 
PISA or the TIMSS are not only producing data for comparisons, they also produce 
conceptions of what is important and of value in Maths Education. They are not only 
comparing, they are participating in the social construction of curricula in Maths 
Education. This thought is well developed in the work of Ian Flacking (1999). From 
this point of view, international comparisons are about homogenisation of Maths 
Education. Second, going back to the correlations between different variables that are 
sought in international comparisons we find another thought and that is that given a 
certain correlation it is predicted that some fact will have an impact on another fact. 
Given what we know about correlations on one side and explanations on the other 
side such conclusions are of course problematic. But we think that, on a pragmatic 
level, even the search for correlations between facts is problematic. What we find is 
an instrumentalistic system of reason, that construct technical directives (von Wright, 
1972) based on abstract numerical relations instead of e.g. didactical arguments. 
Stated otherwise, what is compared in international comparisons of preconditions and 
outcomes are educational phenomena shrunk to fit an instrumentalistic system of 
reason. In a word, international comparisons carried out in this way are examples of 
intellectual thrift of content as well as of educational reason. 
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VIDEO SURVEYS: HOW THE TIMSS STUDIES DREW ON THE 
MARRIAGE OF TWO RESEARCH TRADITIONS AND HOW 
THEIR FINDINGS ARE BEING USED TO CHANGE TEACHING 

PRACTICE 

Karen Givvin 
LessonLab 

If our aim is to improve performance in our educational systems, we must first obtain 
an accurate picture of those systems as they currently exist. To paint a picture of 
teaching practices in eighth-grade mathematics classes in the United States (and 
elsewhere) we sought to document and describe average teaching experiences, not 
exemplary ones. The approach taken in the 1995 and 1999 TIMSS Video Studies was 
that of a video survey. The marriage of the two research traditions offers a way to 
resolve the tension between anecdotes (visual images) and statistics (Stigler et al., 
2000). Bringing together the two research approaches allowed us to overcome some 
of the limitations of each. This, along with cross-national comparison, helped provide 
a detailed description of “typical” classroom teaching. 

This forum presents an exciting opportunity to look closely at different approaches to 
comparative research on mathematics education. We’ve been challenged by 
conference organizers to focus on what is to be compared in comparative research on 
teaching and learning mathematics, and why. Only then should we focus on how 
comparisons can be done. The idea is that the nature of learning and teaching 
mathematics, as the substance of comparative studies, needs to come before a 
discussion of the means and processes of comparison. Beyond this, the goal is to 
examine what each approach can teach us about improving students’ mathematics 
performance. 

WHAT IS TO BE COMPARED? 

I’m here to provide the perspective that guided the TIMSS Video Studies. The 
question of what is to be compared within the TIMSS Video Studies can be 
addressed at multiple levels. At one level, the goal was to examine “typical” teaching. 
That is, we weren’t interested in documenting a particular approach to teaching nor 
did we set out to examine high- versus low-quality teaching. Likewise, we were not 
interested in the differential effects of teaching on different categories of students. 
What we wanted to capture was simply everyday practice as it is experienced by 
teachers and students in different countries. 

The “whaf ’ question can also be asked in terms of the aspects of the classroom 
lessons we examined. The answer is that we coded for a wide array of variables. The 
variables were chosen and developed by mathematics educators and cultural insiders, 
and were guided by both the literature and the desire to adequately capture what was 
seen in the lessons we collected. The time and manpower we had available allowed 
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US to reliably code more than 60 distinct aspects of the lesson, from codes such as 
interaction pattern, mathematical content activity, and activity purpose, to myriad 
codes about each mathematical problem (e.g., evidence of real life connections, 
graphic representations, procedural complexity, and student choice in solution 
methods), to judgments of student engagement, lesson coherence, and overall quality. 

The “what” question can also be asked in terms of what we intended to describe 
when we reassembled the discrete classroom elements we examined. What we hoped 
for was to be able to describe systems of teaching. Our thinking was that the 
individual codes would come together in a coherent way, with particular codes acting 
to inform others and with a broad set being used to describe and give meaning to the 
system. 

WHY MAKE COMPARISONS? 

Because we regard teaching as a cultural activity we began the study with the 
assumption that many classroom activities would vary little within each country and 
would be so familiar to cultural insiders that they would become invisible (Geertz, 
1984). To describe teaching fully requires exploring it in relation to that seen in other 
countries. Examining different cultures helps us see what is commonplace in our own 
classrooms (Stigler & Heibert, 1999; Stigler, Gallimore, & Heibert, 2000) and being 
forced to explain classroom events (or the absence of particular features) to cultural 
outsiders helps draw our attention to details that are otherwise transparent to us. 
Beyond this, examining practices across cultures can help us discover pedagogical 
alternatives. One might, for example, see unfamiliar ways to pose problems, to 
organize how students work on problems, or for teachers and students to interact. 
Discovering alternatives can in turn lead to a discussion of pedagogical choices. The 
TIMSS Video Studies were conducted with these goals in mind. 

HOW WERE DATA AND RESULTS PRODUCED IN THE TIMSS VIDEO 
STUDIES? 

With some whats and whys behind us, we may turn to how we approached the 
process of comparing mathematics teaching and learning. The approach we took was 
that of a video survey. As with traditional survey methods, and in order to arrive at an 
“average,” we began with large, nationally representative samples. Using a national 
sample provides information about students’ common experiences. It is important to 
know what teaching looks like, on average, so that national discussions of teaching 
focus on what most students experience. The survey quality of the research speaks to 
the theme of this year’s conference: inclusion and diversity. By conducting a national 
sample we made a best effort at capturing the full range of teaching, not intentionally 
limiting what we sampled. By applying to it a wide array of codes, we were poised to 
capture the diversity in teaching within and across countries. 

Unlike traditional survey methods, we didn’t use a questionnaire as our primary data 
source. We instead used video. Videos offer the ability to conduct a detailed 
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examination of complex activities from different points of view. They preserve 
classroom activity so it can be slowed down and viewed multiple times, by many 
people with different kinds of expertise, making possible in-depth descriptions of 
many classroom lessons. The marriage of the two research traditions offers a way to 
resolve the tension between anecdotes (visual images) and statistics (Stigler et al. 
2000 ). 

In the more recent and larger-scale of the two studies, we examined between 50 and 
140 lessons (one per participating teacher) from each of seven countries. The TIMSS 
Video Studies were studies of teaching, so the primary of our two cameras focused 
on the teacher. The second, stationary camera was fixed on students and was, in the 
end, used for classroom analysis only rarely. The videos were supplemented by a 
teacher questionnaire. Items on it were sometimes used to clarify lesson events, but 
the questionnaire was more generally used to round out the picture of teachers in each 
country (e.g., years of experiences, education) and their perceptions of the videotaped 
lesson. 

Not surprising based on the data collection design, the TIMSS Video Studies report 
statistically-based characterizations of the ‘typical lesson.’ For each of the codes 
examined we can explore the frequency of occurrence across lessons (or across 
mathematical problems) by country. Examined singly, the codes provide a fine- 
grained description of classroom practice. Organized by concept, they can paint a 
nuanced picture of teaching in each country - one that can then be compared across 
countries. 

Although we feel strongly about the affordances of our research approach, we 
recognize the limitations of it as well. Foremost is the enormous cost of such an 
undertaking. With respect to the data collection procedure, we can say nothing of 
how teaching plays out over a series of lessons, how teachers of varying competence 
teach or the degree of variance within the practices of competent teachers, or of how 
classroom events are perceived by either the teacher or the students. (Fortunately, for 
some of these goals one can turn to David Clarke’s Learner’s Perspective Study.) The 
design of the TIMSS Video Studies also makes it impossible to make a direct link 
between classroom practice and student achievement. 

WHAT CAN BE LEARNED EROM THIS APPROACH AND HOW CAN 
RESULTS BE USED? 

With this approach, we were able to answer questions such as (1) whether teachers in 
all high-achieving countries teach as those do in Japan, (2) with Japan aside, whether 
teachers in high-achieving countries share a common pedagogy, and (3) what, if any, 
features most higher-achieving countries have in common. With regard to the last 
question, Lindblad and Marten correctly point out that we had difficulty finding 
lesson features that correlate with differences in achievement. There was at least one 
feature, however, that appeared to have such a correlation. I’d like to expand on it 
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and on how we’ve begun to use the finding to make an impact on teaching and 
learning in the United States. 

The code to which I’m referring is called “making connections.” In the TIMSS 1999 
Video Study, it was found that U.S. students, in typical classrooms, rarely had 
opportunities to engage in challenging work during their eighth-grade mathematics 
lessons. Although U.S. teachers posed problems with the potential for rich 
mathematical learning just as frequently as did teachers in the other, higher- 
achieving, countries, they almost never maintained the problems at this conceptual 
level as they were worked on and discussed. 

Taking a detailed look, column B of Table 1 shows that the percentage of problems 
categorized as making connections varied across countries, and even among the high- 
achieving countries. While Japan exceeded all of the other countries on this 
dimension (54%), the presentation of making connection problems in the other high 
achieving countries looked more like that found in the U.S. This suggests it is not 
necessary to present a high percentage of rich problems in a single lesson in order to 
produce high levels of mathematics achievement. 

Table 1. Types of Problem Presentation and Implementation in the TIMSS 1999 

Video Study 





A 


B 


c 


D 


Country 


Average 
TIMSS 1995 
mathematics 
score 


%of 
Problems 
Presented as 
Making 
Connections 


% of Making- 

Connections 

Problems 

Implemented As 

Making 

Connections 


% of Making- 

Connections 

Problems 

Converted to 

Lower-Level 

Problems 


Japan 


581 


54 


48 


52 


Hong Kong 
SAR 


569 


13 


46 


54 


Czech 

Republic 


546 


16 


52 


48 


Netherlands 


529 


24 


37 


63 


Australia 


519 


15 


8 


92 


United 

States 


492 


17 


0 


100 



However, presenting a rich problem is one thing, exploiting it for rich learning 
opportunities is another. The data in Column B considers only how a problem is 
presented, not how the problem is implemented as the lesson unfolds. A making 
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connections problem, for example, might be converted into a using procedures 
problem as a teacher works it through with the class. When we look at how problems 
were implemented, quite a different story emerged. In Column C, we present the 
percentage of making connections problems that were actually solved by making 
connections (i.e., not transformed into using procedures, stating concepts, or giving 
results only problems). 

Comparing Columns B and C reveals a striking difference. The data in Column C 
reveal something that high-achieving nations have in common. It is not the 
percentage of rich problems presented but the way they are implemented in the lesson 
that distinguishes them from the United States (and, to some degree, Australia). Most 
of the making connections problems in the United States were converted into lower 
level problems (see Column D). Instead of using these problems as opportunities to 
explore and reason about mathematical concepts, U.S. teachers typically broke them 
into procedural elements and took students through the procedures step-by-step 
(Hiebert, et. ah, 2003). 

This pattern of results can be interpreted in various ways. First, it is possible that U.S. 
teachers lack the content knowledge that would be necessary for them to facilitate 
rich discussions of mathematics (Ma, 1999). Another hypothesis is that U.S. teachers 
have little experience — either as teachers or when they were students themselves - 
engaging in conceptually rich discussions of mathematical problems. Again, we 
argue that teaching is a cultural activity, varying more across cultures than within. If 
this is true, then it will be difficult for teachers to practice instructional strategies that 
are rare in their own culture, and thus less likely that they would have observed many 
examples of others doing so. 

Whatever the merit of these interpretations, the making connections results from the 
TIMSS 1999 Video Study suggest a potentially cost-effective strategy for 
professional development of mathematics teachers. In brief, these data indicate that: 

U.S. mathematics curricula already include rich problems that lend themselves to 
conceptually rich discussions, and that U.S. teachers typically present as many rich 
problems to their students as teachers in high achieving countries; 

Gains in student mathematics achievement might be obtained if U.S. teachers more 
often implemented these rich problems by maintaining their complexity, rather than 
converting them to using procedures problems. 

These results suggest a way to improve mathematics achievement using existing 
curricula and programs, and provide the rationale for our current intervention study. 
With a grant from the Institute for Education Sciences, we are attempting to teach 
teachers to identify and effectively implement mathematically rich problems in their 
pre-algebra lessons. Our plan is to assess the impact of the training on (1) teachers’ 
knowledge of mathematical content for its use in the classroom (i.e., pedagogical 
content knowledge), (2) teachers’ ability to present rich problems in their lessons and 
maintain high conceptual levels of implementation, and (3) students’ mathematical 
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achievement. During the first of two years of implementation, we will compare the 
teachers and students who receive our professional development training with those 
who do not. During the second year of implementation, the control group from the 
first year will receive our PD program as well. The plan will allow us to examine the 
effects of the program on two groups of teachers, as well as enable us to assess its 
effectiveness when used over two consecutive years. 

SUMMARY 

As the pendulum swings to and fro between pedagogical movements and with the 
comings and goings of popular practice and political policy, what often becomes 
overlooked is the need to obtain a clear picture of the state of everyday practiee. The 
absence of such an understanding prevents programs from being adequately informed 
by teacher and student needs. 

With the TIMSS Video Studies, our interest was to describe everyday teaching across 
different countries via a methodology we refer to as video survey. Results indicated 
possible ways of improving classroom teaching. We’re currently pursuing one of 
those ideas - a focus on implementing problems with a conceptual focus - in our 
current intervention program. 
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LEARNER’S PERSPECTIVE STUDY: DEVELOPING MEANING 
FROM COMPLEMENTARY ACCOUNTS OF PRACTICE 

David Clarke 
University of Melbourne 

By examining mathematics classroom practice over sequences of ten lessons, the 
Learner ’s Perspective Study provides data on the teacher ’s and learners ’ 
participation in the co-construction of the possible forms of participation through 
which classroom practice is constituted. The use of post-lesson video-stimulated 
interviews offers additional insight into participants ’ intentions, actions and 
interpretations. Complementarity of account is possible on at least three levels: 
Between study participants (teacher and students - through both videotaped 
classroom actions and post-lesson reconstructive interviews); Between project 
researchers (through parallel analyses of a common data set); and. Between projects 
(eg LPS and TIMSS video studies). All three are important. 

INTERNATIONAL COMPARISONS OF CLASSROOM PRACTICE 

The practices of classrooms are the most evident institutionalized means by which the 
policies of a nation’s educational system are put into effect. Given this, the classroom 
seems a sensible place to look for explanations and consequences of the differences 
and similarities identified in international comparative studies of curriculum, teaching 
practice, and student achievement (see Clarke, 2003). 

If we are to engage in international comparative research, there are two quite distinct 
methodological alternatives: 

Alternative 1. 

If two groups of objects are to be compared then one approach is to consider these 
two questions: 

Difference - “What is the characteristic about which the comparison is to be made?” 

Similarity — “How might each group of objects be separately typified with respect to 
that characteristic?” The international comparison of national norms of student 
achievement could be described as conforming to this approach. 

The order in which these two questions are posed is a major methodological 
signature. 

Alternative 2. 

If two groups of objects are to be compared, consider these two questions: 

Similarity - “Which characteristics appear to typify this collection of objects?” 

Difference - “What comparisons can be made between these two groups of objects 
using the identified characteristics?” 
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Posing the questions as in Alternative 2 reduces the danger of constraining the data to 
a predetermined structure, but may lead to the typification of the two groups by 
different emergent characteristics, restricting the common bases on which 
comparison of the two groups might be made. Note: Alternative 2 assumes a domain 
within which comparison is sought, such as classroom practice or curricular policy. 

For example, it might be that for one nation or culture there is no nationally 
characteristic structure to the lesson as a whole, but that particular types of 
idiosyncratic lesson events offer the most appropriate typification. For another nation 
or culture, there could be a high degree of regularity to the composition of lessons, or 
in the sequencing of particular types of instructional activity in the delivery of a 
topic. Such differences in the form of typification provide a basis for international 
comparison that reflects something more essential to each than the identification 
(imposition) of the same structural level as the basis for the comparison. The 
methodological choice of Alternative 1 makes the basis for comparison a matter of 
prescription based on either theory or on the prevailing educational priorities of the 
country conducting the study. Choice of Alternative 2 makes the identification of 
possible bases for comparison an empirical result of the research. 

DATA IN THE LEARNER’S PERSPECTIVE STUDY 

The Learner’s Perspective Study documented sequences of ten lessons, using three 
video cameras, and supplemented by the reconstructive accounts of classroom 
participants obtained in post-lesson video-stimulated interviews, and by test and 
questionnaire data, and copies of student written material (Clarke, 1998, 2001, 2003). 
In each classroom, formal data collection was preceded by a one-week familiarization 
period in which the research team undertook preliminary classroom videotaping and 
post-lesson interviewing until such time as the teacher and students were accustomed 
to the classroom presence of the researchers and familiar with the research process. In 
each participating country, the focus of data collection was the classrooms of three 
teachers, identified by the local mathematics education community as competent, and 
situated in demographically different school communities within the one major city. 
For each school system (country), this design generates a data set of 30 ‘well-taught’ 
lessons (three sequences of at least ten lessons), involving 120 video records, 60 
student interviews, 12 teacher interviews, plus researcher field notes, test and 
questionnaire data, and scanned student written material. Data collection is complete 
in Australia, China, Germany, Hong Kong, Japan, the Philippines, South Africa, 
Sweden, and the USA; underway in Israel and Korea; and planned for the Czech 
Republic, England and Singapore. The teacher and student interviews offer insight 
into both the teacher’s and the students’ participation in particular lesson events and 
the significance and meaning that the students associated with their actions and those 
of the teacher and their classmates. 
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COMPLEMENTARITY AS ESSENTIAL 



Complementarity Between Partieipant Aeeounts: Establishing the Co- 

Construction of Classroom Practice 

Like Wenger (1998), Clarke’s (2004) analysis of patterns of participation in 
classroom settings stresses the multiplicity and overlapping character of communities 
of practice and the role of the individual in contributing to the practice of a 
community (the class). Clarke (2001) has discussed the acts of interpretive affiliation, 
whereby the learners align themselves with various communities of practice and 
construct their participation and ultimately their practice through a customizing 
process in which their inclinations and capabilities are expressed within the 
constraints and affordances of the social situation and the overlapping communities 
that compete for the learner’s allegiance and participation. By examining sequences 
of ten lessons, the Learner’s Perspective Study provides data on the teacher’s and 
learners’ participation in the co-construction of the possible forms of participation 
through which classroom practice is constituted (cf Brousseau, 1 986). An example of 
utilizing the complementarity of teacher and student accounts can be found in Clarke 
(2004), which examines the legitimacy of the characterisation of kikan-shido 
(Between-Desks-Instruction) as a whole class pattern of participation, and to situate 
the actions of teacher and learners in relation to this pattern of participation. By 
drawing on classroom video evidence and juxtaposing teacher and student interview 
data, it is possible to demonstrate that while engaging in kikan-shido, the teacher and 
the students participate in actions that are mutually constraining and affording, and 
that the resultant pattern of participation can only be understood through 
consideration of the actions of all participants. A key characteristic of kikan-shido, as 
it is practiced in the Australian classrooms, is the implicit devolution of the 
responsibility for knowledge generation from the teacher to the student, while still 
institutionalizing the teacher’s obligation to scaffold the process of knowledge 
generation being enacted by the students. Comparison with the enactment of kikan- 
shido in other classrooms (Hong Kong, Shanghai, and San Diego, for example) 
provides significant insight into the pedagogical principles underlying the practices of 
different classrooms internationally. 

Complementarity Between LPS Researcher Acconnts: A More Comprehensive 
Portrayal of Classroom Practice 

Classrooms are complex social settings, and research that seeks to understand the 
learning that occurs in such settings must reflect and accommodate that complexity. 
This accommodation can occur if your data collection process generates a sufficiently 
rich data set. Such a data set can be adequately exploited only to the extent that the 
research design employs analytical techniques sensitive to the multifaceted and 
multiply-connected nature of the data . . . we need to acknowledge the multiple 
potential meanings of the situations we are studying by deliberately giving voice to 
many of these meanings through accounts both from participants and from a variety 
of “readers” of those situations. The implementation of this approach requires the 
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rejection of consensus and convergence as options for the synthesis of these accounts, 
and instead accords the accounts “complementary” status, subject to the requirement 
that they be consistent with the data from which they are derived, but not necessarily 
consistent with each other, since no object or situation, when viewed from different 
perspectives, necessarily appears the same (Clarke, 2001, p. 1). In the LPS project, 
multiple, simultaneous analyses are being undertaken of the accumulated 
international data set from a variety of analytical perspectives. For example, while 
Ference Marton and his colleagues in Sweden and Flong Kong analyse the practices 
of classrooms in Shanghai from the perspective of Marton’s Theory of Variation, 
Clarke is undertaking analysis of the same lessons in relation to the Distribution of 
the Responsibility for Knowledge Generation. These two analytical approaches do 
not appeal to the same theoretical premises, but nor are they necessarily in conflict. 
They represent complementary analyses of a common body of data, aspiring to 
advance different theoretical perspectives and to inform practice in different ways. 

Complementarity Between Project Acconnts: Approaches to Studying Lesson 
Structure 

Lesson structure can be interpreted in three senses: 

- At the level of the whole lesson - regularity in the presence and sequence of 
instructional units of which lessons are composed; 

- At the level of the topic - regularity in the occurrence of lesson elements at points 
in the instructional sequence associated with a curriculum topic, typically lasting 
several lessons; 

- At the level of the constituent lesson events - regularity in the form and function of 
types of lesson events from which lessons are constituted. 

A research design predicated on a nationally representative sampling of individual 
lessons, as in the TIMSS Video Studies (1995 and 1999), inevitably reports a 
statistically-based characterization of the representative lesson (the first of the 
alternatives listed above). The analysis of video data collected in the first TIMSS 
video study (Stigler and Fliebert, 1999) centred on the teacher’s adherence to a 
culturally-based “script.” Central to the identification of these cultural scripts for 
teaching were the “lesson patterns” reported by Stigler and Fliebert for Germany, 
Japan and the USA, and the contention that teaching in each of the three countries 
could be described by a “simple, common pattern” (Stigler & Fliebert, 1999, p. 82). 

The characterisation of the practices of a nation’s or a culture’s mathematics 
classrooms with a single lesson pattern has been problematised by the results of the 
Learner’s Perspective Study (see www.edfac.unimelb.edu.au/DSME/lps ). The recent 
report of the TIMSS 1999 Video Study (Fliebert et ah, 2003) employed ‘lesson 
signatures’ rather than ‘lesson patterns’ to characterize differences between the 
practices of international mathematics classrooms internationally. These lesson 
signatures characterize national norms of practice in terms of the prevalence of 
different activity types at different points in the lesson. The resultant ‘signatures’ 
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remain insensitive to the location of the sampled lesson(s) within a topic sequence. 

As such, they can give a misleading impression that the structure of any particular 
lesson is independent of whether it is the introductory lesson at the commencement of 
a topic, a consolidation or developmental lesson later in the topic sequence, or a 
summative lesson occurring towards the end of a topic. Nonetheless, the TIMSS data 
offers the opportunity to estimate the prevalence of a particular activity type 
identified as significant from LPS data. Similarly, activities identified in the TIMSS 
project as prevalent within a particular country can be evaluated from within the LPS 
data in relation to their capacity to stimulate specific responses in students, 
particularly learning outcomes. The complementarity of these two projects is 
acknowledged and valued by both research groups. 

CONCLUDING REMARKS 

This paper has offered complementarity of accounts as an essential methodological 
and theoretical stance, adopted by the Learner’s Perspective Study, for the 
explication of mathematics teaching and learning in classroom settings, the 
advancement of theories relating to such settings, and the informing of practice in 
mathematics classrooms. This paper and the Research Forum of which it is a part 
embodies the PME conference theme of ‘inclusion and diversity’ in a very 
fundamental way. 
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LESSONS FROM A SMALL-SCALE OBSERVATIONAL STUDY: 
AN EXAMPLE OF THE TEACHING OF FRACTIONS 

Ulla Runesson 



Goteborg University, Sweden 
Ida Ah Chee Mok 
The University of Hong Kong 

The aim of this paper is to demonstrate what a small-scale project can tell about 
features of teaching and learning in two different cultures. We argue that some 
features, which may not be easily observed within one culture, can become more 
visible in the contrast in order to get a better understanding of the teaching practice 
per se, even from a small scale project. We have studied the mathematics teaching in 
one classroom in Hong Kong and four in Sweden. Based on the assumption, that how 
the content is taught has an important implication on what students may possibly 
learn, we compared how the teaching of the same topic (fraction) may differ between 
the two places. Some profound differences regarding how the same topic was dealt 
with in the two countries were found. In the Hong Kong data several things were 
handled in one lesson at the same time whereas in the Swedish data this happened in 
a sequence of lessons spreading over a substantial period. 

INTRODUCTION 

Being in an environment constantly, one usually takes things for granted and fails to 
see the characteristics of the environment as special or different from the others. To 
bring about a better understanding of mathematics teaching itself is one argument for 
comparative studies (Stigler & Hiebert, 1999; Lopez-Real & Mok, 2002). However, 
comparison can be made at different levels and with different focus. Mostly, these 
studies are to different extent grounded in data from more extensive data sets (e.g. 
TIMSS Video Study and in the PISA-project). But, are these very expensive and 
extensive studies the only way to bring about insights about cultural differences? The 
study we will report here captures a small number of mathematics lessons in Hong 
Kong and Sweden. In Sweden five consecutive lessons from four different 
classrooms and in Hong Kong only one lesson were studied. Compared to the 
extensive studies mentioned, our study can appear to be too thin and insufficient to 
generalize anything about mathematics teaching in the two countries. However, our 
aim was to some extent different from these studies the overall aim of which was to 
compare the teaching practices in different cultures. This was not the goal for our 
study. Instead we hoped that some features, which may not be easily observed within 
one culture, would become more visible in the contrast in order to get a better 
understanding of the teaching practice per se. The question whether this is possible 
even within the frame of a small-scale project will be discussed in this paper. 
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THEORETICAL FRAMEWORK: A THEORY OF VARIATION 



In this study, we approach classroom learning with a specific focus. Assuming 
learning is always learning of something — it has an object - we study how the object 
of learning is constituted in the classroom interaction, and with the particular interest 
in different possibilities to learn in different situations. What is possible to learn, has 
to do with those aspects of the object of learning that are possible for the learners to 
be aware of, or to discern. However, only that which is varying can be discerned 
(Bowden & Marton, 1 998). So, the possibility to discern an aspect has to do with 
whether it is present as a dimension of variation or not (Marton & Booth, 1997; 
Marton, Runesson, & Tsui, 2004). If the particular aspect is present as a dimension of 
variation, it is likely discerned by the learner. And further, if the aspects are present 
as dimensions of variation at the same time, the learners likely discern them at the 
same time. So, what is studied is the pattern of simultaneous dimensions of variations 
related to the object of learning that are present to the learners in the situation 
(Runesson & Marton, 2002). And when studying the differences in possibilities to 
learn in different classrooms, it is the difference between the patterns of simultaneous 
dimensions of variations opened in the different classrooms that we describe. 

THE STUDY 

The current study has its origin in a previous study of Swedish mathematics 
classrooms, which aimed at finding differences between the teachers as regards how 
the topic was handled (Runesson, 1999). To shed new light on this data, a similar 
study in Hong Kong was conducted. The aim was to find differences between how 
the same topic was taught by contrasting mathematics teaching in two different 
cultures. However, to be able to see critical features in our own classrooms and one’s 
own culture, we chose the same mathematical topic in order to see how the same 
topic can be handled in different cultural context. Therefore, the selection of the 
Hong Kong data set was made on the basis of matching up with the existing Swedish 
data as much as possible. The Hong Kong lesson was a primary four (age 10, grade 
4) lesson on the topic “Comparing fractions”. The lesson was carried out in 
Cantonese and videotaped. The Swedish data is drawn from a larger data set 
consisting 20 lessons from four different classrooms in grade six and seven. These 
lessons were audio taped and transcribed verbatim. Our aim was to be as close as 
possible with regards to the content of teaching. That is, when sections of the 
Swedish data were selected, this was done at the level of sub-constructs of fractions. 
The sub-constructs of fractions, which were available in the Hong Kong data, did 
appear in four of Swedish teachers' teaching. The analysis is grounded on data from 
all of these classrooms. Due to differences between the Swedish and the Hong Kong 
curriculum, we could not match the age of the pupils in the two countries. The topic 
was taught in grade six and seven (age 12 and 13 respectively) in Sweden and in 
grade four in Hong Kong (age 10). And although, we tried to come as close as 
possible to study the same content, some differences occurred. In the Hong Kong 
lesson the students worked with finding the common denominators of two fractions. 
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In the Swedish lessons the tasks was slightly different; the task was to find another 
fraction with the same value (e.g. 2/6=l/3). However, in both the Hong Kong and the 
Swedish lessons, comparison of fractions with different denominators was found. 
Unlike the Hong Kong data, which is drawn from one single lesson, the Swedish data 
consist of several lessons. 

SUMMARY OF THE RESULTS: TWO DIFFERENT EXAMPLES OF 
SIMULTANEITY AND VARIATION 

The analysis was with a particular focus on those aspects of the topic taught that were 
opened as dimensions of variation were identified. The Hong Kong lesson appears to 
have only one objective, i.e. comparing fractions with different denominators. 
Nevertheless, this objective was visited and revisited via several tasks, which were 
either in the form of questions in the worksheets or examples on the board. As a 
result of this, the Hong Kong lesson shows a pattern of variation, which consists 
many dimensions of variation. For example, some dimensions are: alternative 
representations of the method of amplification, the denominators, the fractional parts 
of different wholes and the contrast between the methods of comparison. Moreover, 
the intertwined relationship between these dimensions of variation forms a special 
arrangement or simultaneity of variation in a single lesson. Such experience is 
important because it provides a chance for “fusion” i.e., for the students to consider 
several aspects of the object of learning simultaneously (Marton, Runesson and Tsui, 
2004). The Swedish lessons showed a very different pattern of variation. The most 
striking difference was perhaps that variation of methods was not opened. The 
students were presented to one method only, a diagrammatic method. Instead of 
varying the method, the teacher demonstrated a method on a couple of different 
examples. The other apparent difference was the sequential character identified in the 
Swedish lessons. We found that these sub-constructs were commonly never presented 
simultaneously in the Swedish lessons, but instead they were extended over time and 
presented as disjoint instances without any connection or reference to previous 
lessons. So, finding the bigger of two different fractions was taught in one lesson, and 
"fractions with different denominators but with the same value" was taught in 
another. The latter was taught with no reference to how this had been presented 
earlier although the two topics were indeed connected. In other words, since in the 
Hong Kong lesson several sub-constructs were presented and related to each other at 
the same time, the Hong Kong lessons were richer in terms of sub-constructs related 
at the same time. 

Comparing to the Hong Kong lesson, the Swedish examples created a narrower space 
of variation, and in combination with the sequential character, accomplished a quite 
different space for learning in the Swedish lessons. From the theoretical position 
taken, we can assume that what was possible to discern of the same thing was 
different in Hong Kong and in Sweden. In other words, the students’ understanding 
of the two sub-constructs “comparison of fractions” and “fractions with the same 
value” are very likely to be different when the students from the two places 
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experience such different space of learning. So, what we can say is, it was possible to 
discern different things in Hong Kong and Sweden. But, what that means for what the 
students actually learned, we cannot say, since this has not been studied. 

WHAT COULD COME OUT OF A SMALL-SCALE OBSERVATIONAL 
STUDY? 

The study presented here is in many respects a small-scale project, so what could 
possibly come out of such, as it seems, limited project? 

The original purpose of this study was to shed new light on a study conducted earlier 
in Swedish classrooms. In line with the theoretical framework taken, discovering 
something new when revisiting the data would be easier if it was contrasted against 
something different, e.g. by contrasting mathematics classrooms and possibilities to 
learn in different school systems and educational traditions. The object of research in 
this study was not possibilities to learn in a general sense, but possibilities to learn the 
same thing. Therefore, it was important to study how the same topic was dealt with, 
i.e. to keep the content constant. This design has been used in a number of studies 
(Runesson, 1999; Marton & Morris, 2002; Marton, Tsui et ah, 2004) However, it was 
in many ways a bit problematic to match up with a data set from Hong Kong to the 
existing Swedish data. From our point of view we wanted to delimit our definition of 
“the same topic” as much as possible. “The same topic” was defined in terms of how 
it appears in classroom practice, and on the level of tasks, so we asked the teacher to 
invite us to study a lesson when fractions with different denominators would be the 
topic taught. Although, we tried to come as close as possible to study the same 
content, some differences occurred. Being restrictive to having the same topic, it was 
not possible to study pupils of the same age, due to different curricula in the two 
countries. However, from this point of view the result is interesting. In the Hong 
Kong classroom the pupils were about three years younger than their counterparts, 
however a space of learning consisting of many simultaneous dimensions of variation 
was afforded to the learners, whereas for the older Swedish pupils dimensions of 
variation were brought out in sequence. 

It could be argued that this sequential pattern of variation was a result of the longer 
period of observed lessons in the Swedish data, that the likelihood of such a finding is 
bigger if several consecutive lessons are observed. It could not be excluded that the 
sequential character of handling the object of learning, which was found in the 
Swedish data, would not appear in Hong Kong. This has never been claimed, and it 
was never the purpose of the study either, i.e. to say anything about the general in the 
two cultures. What we have described is two different ways of handling the same 
topic, or two different patterns of variation and simultaneity when teaching the same 
topic. This was found by comparing two different school cultures. 

The way we worked in this study, implied doing a close and narrow analysis, but 
without the aim of finding more overall patterns or a more general character in the 
different classrooms. A main difference between, for instance, the TIMSS Video 
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Study and ours is what we were studying. To us the TIMSS Video Study was a study 
of teaching, whereas ours is a study of possibilities to learn the same thing. In our 
study we identified and described how the same object of learning could be dealt with 
differently by means of examples from different cultures. This was possible to do, 
even if only one single lesson from one teacher from each country was studied. 

Necessarily a small-scale project like this touches the issue of representatives. Our 
aim was not to come up with something that could tell us something about the 
possibilities to learn about fractions with different denominators in Swedish and 
Hong Kong classrooms in general, or to explain differences in learning outcomes 
between the two countries. Instead we wanted this study to open our eyes to that, 
which is not easily seen within our own culture, so it would become visible, but 
without saying anything about the typical Swedish or the typical Hong Kong 
classroom. The most prominent coming out of this study is, that by seeing what could 
be done differently, what could be the case, new light has shed light on what is done 
and what is the case in some classrooms our own countries. When the characteristics 
identified from the two different data sets are used as a mirror, it gives us a better 
understanding of the practice in our countries. Surprisingly, such understanding could 
be achieved from a small-scale study like this. 
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THE PISA-STUDY: DIFFERENTIATED ASSESSMENT OF 
“MATHEMATICAL LITERACY“ 

Michael Neubrand 

Carl-von-Ossietzky-University Oldenburg 

More than some international comparative studies before, PISA (Programme for 
International Student Assessment) referred to conceptions from Mathematics 
Education. Specific backgrounds of PISA are 

- the ideas behind realistic mathematics education as formulated by Freudenthal and 
de Lange, and 

- in the German national option of PISA, the distinction of cognitive activities 
required in the item, esp. conceptual vs. procedural thinking (Hiebert). 

According to that theoretical basis, PISA gives insight into the structure of 
mathematical achievement. These “profiles ” bear messages which come closer to the 
needs of the development of mathematics education in the countries. 

INTENTIONS OF THE INTERNATIONAL PISA-STUDY 

PISA, the “Programme for International Student Assessment” is a study initiated by 
the Organization for Economic Cooperation and Development (OECD). PISA 
compares the achievement of 15 years old students in the (industrialized) OECD- 
countries in the domains Reading, Mathematics and Science (OECD 1999). The first 
test was held in 2000, followed by tests in 2003 and 2006. See OECD (2001) for 
results of PISA-2000; PISA-2003 will be released in Dec 04. 

That PISA tests an age based sample of students, but not a grade based sample as 
many other studies did, has its origin in the political intentions of PISA. The OECD 
wanted to gain insight into the outcomes of the educational systems in the countries. 
Therefore, to choose the 15 years olds was decided according to the idea that this is 
the age of transition to vocational training or to an extended secondary school career. 
However, this decision once made has the consequence that a “core-curriculum” 
approach, as e.g. TIMSS has chosen, should not be appropriate. In apparent contrast, 
PISA focused on “mathematical literacy” which is intended to target the resulting 
abilities acquired in school up to the age 15, from whatever schooling it may come. 

Mathematical literacy focuses on the “functional use of mathematics” (OECD 1999) 
in various situations, not only realistic ones, or as the framework says: “Mathematical 
literacy is an individual’s capacity to identify and understand the role that 
mathematics plays in the world, to make well-founded mathematical judgments, and 
to engage in mathematics, in ways that meet the needs of that individual’s current and 
future life as a constructive, concerned and reflective citizen.” (OECD 1999) 
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MEETING THE FULL RANGE OF MATHEMATICS 



Any serious test in mathematics or in other domains has first to clear up its views of 
the area tested, mathematics in our case. A literacy test as PISA is even stronger 
urged to do this. The PISA mathematics items are thus constructed as problems 
which call for using mathematics in context, i.e. mathematical modelling in a very 
broad sense, including not only contexts from daily life, but also components of the 
social and cultural embedding of mathematics (OECD 1999). It is not the place here 
to go into a deeper discussion of what “literacy” should mean: see e.g. Kilpatrick & 
al. (2001), Jablonka (2003), Neubrand (2003) for an extended discussion. However, 
one common idea is a kind of red thread through all these discourses: “literacy” is not 
fully captured if one does not try to meet the full range of mathematics. 

A starting point for considerations of what mathematical activity can be is the well 
known picture of the (idealized) process of modelling (Fig.l): 



Figure 1 : The Process of Mathematical Modelling 
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This picture can be applied to the modelling of situations from outside of 
mathematics, as it was created for, but it may also describe mathematical problem 
solving when the situation one starts from is a challenging problem inside of 
mathematics. From the cognitive point of view, both activities have the same 
structure: Modelling in applications as well as working on intra-mathematical 
problem bearing situations is essentially characterized by the activity of 
transformation, translation, seeing in other connections etc., which is called 
mathematization in the classical sense, but equivalent to the activity of structuring an 
intra-mathematical situation. 

The process of “working out” as in Fig. 1 can follow two cognitive ways: We distinct 
if the working-out process of an item needs more procedural or more conceptual 
thinking (Hiebert 1986). This idea was added to the international framework of PISA 
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in the German national option to PISA -2000 (Neubrand et al 2001). Such a 
distinction essentially defines three classes of items (Knoche & al 2002) which we 
called „ The Three Types of Mathematical Activity 

(a) employing only techniques (abbreviated: „technical items"), 

(b) modelling and problem solving activities which lead to use mathematical tools 
and procedures (abbr.: items with „procedural modelling") 

(c) modelling and problem solving activities which need drawing connections and 
using mathematical conceptions (abbr.: „conceptual modelling"). 

What "technical items” are is very clear. These items consist of just "working out” in 
the picture of Fig. 1 , and the whole lower half-plane in the picture does not even exist 
in that item. Regretfully, all too often knowing mathematics is considered as just to 
be able to run an algorithm when the starting point is given. All non-technical items, 
i.e. those items in which the lower half-plane in Fig.l is relevant, are called here 
"modelling problems”. Among the "modelling tasks”, two kinds of "working out” (in 
the sense of Fig.l) make the difference from cognitive perspective: "Working out” 
can consist of solving an equation, doing a calculation, etc., to produce the result, as 
it is the case in the most "classical” textbook problems. These items are the 
„procedural modelling items”. In contrast, there are items, which can be solved by 
giving a appropriate argument, or by connecting the situation to a mathematical 
concept and drawing conclusions from that connection, etc. In this cases we speak of 
„conceptual modelling" items. 

The "full range of mathematics” is therefore defined by giving items from all three 
types of mathematical activity. It is this distinction between different aspects of 
mathematical thinking which rules the German national framework for the national 
option of PISA in Germany (Neubrand & al. 2001). 

THE IDEA OF “PROFILES” OF MATHEMATICS ACHIEVEMENT 

There is empirical evidence, that in some cases a differentiated analysis of 
mathematical achievement gives insight into the inner structure of mathematical 
achievement - and is possible on the basis of a full set of items distributed over the 
three Types of Mathematical Activity. We sketch only two instances of such 
"profiles” (cf Neubrand & Neubrand 2004 for details). 

(A) Individual differences on the three types of mathematical activities 

Knoche & al. (2002) showed this scatter plot (Fig. 2). At least in the German 
population of PISA, there is not enough reason to conclude that good performance in 
the technical items is sufficient to show good performances on the conceptual 
modelling items. 
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Figure 2: Achievement of German students on “technical” and “conceptual” items 
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(B) Differences between conntries 

The following picture (Neubrand & Neubrand 2004) shows the striking different 
behaviour of Japanese and Finish students in PISA. Against the achievement in the 
OECD average (the diagonal line) there are plotted the means for every international 
item in the respective country. 

Figure 3: Different achievement of two high scoring countries, Japan and Finland 
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Apparently, the Japanese population in PISA behaves very different from both, the 
OECD average, and one other high achieving country, Finland. Furthermore, Finland 
shows a tendency to score very well at the easier items, but does not show substantial 
advantages at the harder items. 
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CONCLUSION: STRUCTURES OF ACHIEVEMENT POINT TO 

DIFFERENCES IN TEACHING AND LEARNING 



Comparisons of mathematical achievement can, or at least should, used to indicate 
fields of further development of mathematics education. However, this requires a 
broad and theoretically based picture of the mathematics included in the test. In PISA 
this reflections were done extensively. The results shown here indicate empirical 
foundations e.g. that it is not a promising way to restrict mathematics education to a 
better performance of technical abilities (Fig. 2), nor that “high achievement” can be 
restricted to the one score point on the “horse race axis” (Fig. 3). In both cases, the 
inner structure of mathematical achievement revealed areas where to act, in Germany 
on the improvement of conceptual capabilities, in Finland on stronger results also at 
the more challenging items. 
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